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Abstract 

The Unbounded Knapsack Problem (UKP) is a well-known variant of the famous 0-1 Knapsack 
Problem (0-1 KP). In contrast to 0-1 KP, an arbitrary number of copies of every item can be 
taken in UKP. Since UKP is NP-hard, fully polynomial time approximation schemes (FPTAS) 
are of great interest. Such algorithms find a solution arbitrarily close to the optimum OPT(/), 
i.e. of value at least (1 — e)OPT(/) for e > 0, and have a running time polynomial in the input 
length and U For over thirty years, the best FPTAS was due to Lawler with a running time 
in 0{n -b and a space complexity in 0{n -I- ^), where n is the number of knapsack items. 
We present an improved FPTAS with a running time in 0{n -I- ^ log^ and a space bound in 
0(n + ^ log^ j). This directly improves the running time of the fastest known approximation 
schemes for Bin Packing and Strip Packing, which have to approximately solve UKP instances 
as subproblems. 


1 Introduction 

An instance I of the Knapsack Problem (KP) consists of a list of n items ai,..., an, n £ N, where 
every item has a profit pj £ (0,1] and a size Sj £ (0,1]. Moreover, we have the knapsack size c = 1. 
In the 0-1 Knapsack Problem (0-1 KP), a subset V C {ai,..., Un} has to be chosen such that the 
total profit of V is maximized and the total size of the items in V is at most c. Mathematically, the 
problem is defined by m.ax{J2]j=i PjXj\ J2'j=i ^ c; Xj £ {0,1} VJ}. In this paper, we focus on 
the unbounded variant (UKP) where an arbitrary number of copies of every item is allowed, i.e. we 
want to determine max{J2^=iPjXj\ ^jXj < c; Xj £ N Vj}. 


1.1 Known Results 


The 0-1 Knapsack Problem and other variants of KP are well-known NP-hard problems [^. They 
can be optimally solved in pseudo-polynomial time by dynamic programming [T 18 . Furthermore, 
fully polynomial time approximation schemes (FPTAS) are known for different variants of KP. An 
FPTAS is a family of algorithms (Ae)£>o, where for every e > 0 the algorithm finds for a given 
instance I a solution of profit A^{I) > (1 — e)OPT(/). The value OPT(/) denotes the optimal value 
for I. FPTAS have a running time polynomial in 4 and the input length. 


’Research supported by DFG project JA612/14-2, “Entwicklung und Analyse von efflzienten polynomiellen 
Approximationsschemata fiir Scheduling- und verwandte Optimierungsprobleme” 
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The first FPTAS for 0-1 KP was presented by Ibarra and Kim with a running time in 
0(nlogn -|- ^ • min{^ log(^),n}) and a space complexity in 0(n + ^). Lawler 21 improved the 


running time to 0 (t -|- log(Mn). In 1981, Magazine and Oguz 23 presented a method to decrease 


the space complexity of the dynamic program so that their FPTAS runs in time 0(n^ log(n)^) and 

needs space in 0(f). (The paper focuses on the improved space complexity without a partitioning 

and reduction of the items as done e.g. by Lawler. Without it, Lawler’s basic algorithm has in 

2 

fact a time and space complexity in 0(^).) The currently fastest known algorithm is due to 

lOl |l7l |l8l pp. 166-183] with a space bound in 0(n -f and a running 
■ log(i) ■ min{n, ^ log(^)}). Assuming that n G n(^ log f), this is 


og - 


Kellerer and Pferschy 
time in 0(n minjlogn, 
in 0(nlog(i) -f ^log2(i)). 

For UKP, Ibarra and Kim presented the first FPTAS by extending their 0-1 KP algorithm. 
Their UKP algorithm has a running time in 0(n + ^ log f) and a space complexity in 0{n -f ^). 
Kellerer et al. |18[ pp. 232-234] have moreover described an FPTAS with a running time in 
0(nlog(n) + ^{n + log ^)) and a space bound in 0{n + ^)- In 1979, Lawler 21 presented his 


FPTAS with a running time in 0{n -\- fj) and a space complexity in 0{n+ ^). For n G this 
is still the best known FPTAS. 

The study of KP is not only interesting in itself, it is moreover motivated by column generation 
for optimization problems like the famous Bin Packing Problem and Strip Packing Problem. In 
the former problem, a set J of n items of size in (0,1] has to be packed in as few unit-sized bins as 
possible. In the latter problem, a set J of n rectangles of width (0,1] and height (0,1] has to be 
packed in a strip of unit width such that the height of the packing is minimized. Many algorithms 
for optimization problems like Bin Packing have to solve linear programs (LPs), but enumerating all 
columns of the linear programs would take too much time. One way to avoid this is the consideration 
of the dual of the LP and to (approximately or exactly) solve a separation problem, e.g. KP, to find 
violated inequalities of the dual. These inequalities correspond to columns in the primal LP: the 
columns needed for solving the LP are therefore generated and added dynamically. Examples can 
be found in [6 14 


Since Bin Packing and Strip Packing are NP-complete |^, several approximation algorithms 
have been found for both problems. However, no efficient (i.e. polynomal-time) algorithm A for BP 
or SP can achieve A{J) < c • OPT(J) for c < | and all problem instances J unless P = NP j^: we 
have c > I for the absolute approximation ratio c. The bound | is due to the fact that a polynomial 
algorithm could otherwise distinguish between the optimum of 2 or 3 for BP instances and therefore 
solve the NP-complete Partition Problem in polynomial time [^. Since only such small instances 
prevent an absolute ratio better than |, larger instances may allow for a better approximation ratio. 

So-called asymptotic fully polynomial-time approximation schemes (AFPTAS) (A£)e>o are 
therefore especially interesting. They find for every e > 0 and instance J a solution of value at most 
(1 + e)OPT(J) -|- /(^), and have a running time polynomial in the input length and Roughly 
speaking, AFPTAS achieve an approximation ratio of c = (1 -f e) for large problem instances. 

For Bin Packing, the hrst AFPTAS was presented by Karmarkar and Karp 14 with /(-) = 

1 1 ^ 
O(^). In 1991, Plotkin et al. 24 described an improved algorithm with a smaller additive term 


f{-) = 0{- log(-)) and running time in 0{^ log®(-) -f log(-)n). The AFPTAS by Shachnai and 
Yehezkely 25 has the same additive term and a running time in 0(^ log'’^(-) -f log(-)n) for general 
instances. Currently, the AFPTAS in 10 has the smallest additive term /(-) = 0(log^ -) and the 


fastest running time in 0(i log ^ + log(^)n). 
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The first AFPTAS for Strip Packing was presented by Kenyon and Remila 
Bougeret et al. and Sviridenko 


26 


19 


with f{l) = 0(4). 


independently improved the additive term to /(j) = 0(^ log ^). 
The algorithm in needs time in 0(^log(^) + nlogn), which is the currently fastest known 
AFPTAS. 

Both algorithms in [2 10 solve UKP instances for column generation. A faster FPTAS for UKP 
therefore directly yields faster AFPTAS for Bin Packing and Strip Packing. 


1.2 Our Result 

We have derived an improved FPTAS for UKP that is faster and needs less space than Lawler’s 
algorithm. 

Theorem 1. There is an FPTAS for UKP with a running time in 0{n + ^ log^(^)) and a space 
complexity in 0{n + ^ log^(^)). 

Not only the improved running time, but also the improved space complexity is interesting 
because “for higher values of - the space requirement is usually considered to be a more serious 
bottleneck for practical applications than the running time” [T^ p. 168]. Nevertheless, the improved 
time complexity has direct practical consequences. Let KP(d,£) be the running time to find a 
(1 — e) approximate solution to a UKP instance with d items. The Bin Packing algorithm in [10| 
has the running time 0{KP{d, |) • ^ log ^ + log(^)n) if we assume that KP{d, |) G (where 

£ G 0(e) and d G 0{^ log ^)). By using the new FPTAS for UKP, we get the following result: 

Corollary 2. There is an AFPTAS (A£)e>o for Bin Packing that finds for e G (0, a packing of 
J in A^{J) < (1 + e)OPT(J) + 0(log^(i)) bins. Its running time is in 

o(|,log‘)+log(‘)n) . 

Similarly, the Strip Packing algorithm in (see also j^) has a running time in 0{d{^ + 
In d) max{iLP((i, |), (ilnln(^)} + nlogn) where again d G O(^log^) and e G 0(e). The new 
FPTAS yields the following improved AFPTAS: 

Corollary 3. There is an AFPTAS (Ae)£>o for Strip Packing that finds a packing for J of total 
height A^{J) < (1 + e)0PT(J) + 0{^ log(^)). Its running time is in 

o(ilog‘)+log(t)n) , 

The result in this paper was first presented at IW0CA 2015 [^. The final publication will be 
available at link.springer.com. 

For readers acquainted with column generation or linear programs, it should be noted that the 
LP solved has the form minjc^x | Ax > 6, x > 0}. It is indeed a fractional covering problem where 
the columns of A represent configurations: a configuration assigns item slots to one bin (for Bin 
Packing) or to one shelf of the strip (for Strip Packing) such that the slots fit into the bin or the 
strip. The primal LP is then approximately solved with a method by Grigoriadis et al. (see also 
i). The columns (i.e. configurations) are generated by solving so-called block problems, which are 
UKP instances in this case. When the LP has been solved, each item is placed in a slot that has at 
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least the size of the item. As a feasible solution to the LP has been found, there are enough slots 
for all items. Because of the unboundedness, some configurations may indeed assign more item 
slots of a certain size to the strip or to one or several bins than there are items in the considered 
Strip or Bin Packing instance. This does not represent a problem because the supernumerary item 
slots are simply left empty in the final solution. For comparison, Plotkin et al. [24] solve the LP 
with a decomposition method where the block problem has additional constraints on the knapsack 
variables: it is a Bounded Knapsack Problem where a limited number dj £ N of copies for every 
item aj may be taken. 

1.3 Techniques 

Most algorithms for UKP rely on 0-1 KP algorithms. The 0-1 KP algorithms determine a 

first lower bound Pq for OPT(/). Based on a threshold T depending on Pq, the items are partitioned 
into large (-proht) items with pj > T and small(-proht) items with pj < T. A subset of the large 
items is taken, which is sufficient for an approximate solution. Its prohts are then scaled and the 
well-known dynamic programming by profits applied to the subset. All combinations of large items 
(packed by the dynamic program) and small items (which are greedily added) are checked and the 
best one returned. For UKP, copies of the items in the reduced large item set are taken to transform 
the UKP instance into a 0-1 KP instance. 

Our algorithm also first reduces the number of large items. However, we further preprocess 
the remaining large items by taking advantage of the unboundedness: large items of similar profit 
[2^T, are iteratively combined (“glued”) together to larger items. Apart from two special cases 

that can be easily solved, we prove for this new set G a structure property: there are approximate 
solutions where at most one large item from every interval [2^r, 2^+^r) is used, i.e. only 0(log ^) 
items in total. As a next step, a large item Oeff-c that consists of several copies of the most efficient 
small item is introduced. We prove that there are now approximate solutions to the large items 
GU {oeff-c} of cardinality 0(log ^) and that additionally use at least one item of profit at least \Pq. 
Instead of exact dynamic programming, we use approximate dynamic programming: the profits in 
[jPo) 2Po] are divided into intervals of equal length. During the execution of the dynamic program, 
we eliminate dominated solutions and store for each interval at most one solution of smallest size. 
The combination of approximate dynamic programming with the structure properties yields the 
considerable improvement in the running time and the space complexity. The algorithm then returns 
the best combination of large items (packed by the dynamic program) and copies of the small item 
Oeff (added greedily). 

2 Preliminaries 

We introduce some useful notation. The profit of an item a is denoted by p{a) and its size by s{a). 
If a = aj, we also write p{aj) = pj and s{aj) = Sj. Let V = {xa : a | o £ I, Xa £ N} be a multiset 
of items, i.e. a subset of items in I with their multiplicities. We naturally dehne the total proht 
p{V) := J2xa>0P(^)^a- total size s(U) := Z)a;a >0 

Let u < c = 1 be a part of the knapsack. The corresponding optimum proht for the volume 
V is denoted by OPT(I, u) = max{X)ae/P(®)^« I J2a£i < v; a G P, Xa G N}. Obviously, 

OPT(/) = OPT (/, c) holds. 
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We assume throughout the paper that basic arithmetic operations as well as computing the 
logarithm can be performed in O (1). 

Finally, we have a remark about the use of “item” and “item copy” when we consider a solution 
to a UKP instance. 

Remark 4. Let I, I be two sets of knapsack items with I ^ I. In the 0-1 Knapsack Problem, a 
sentence like “the solution to I uses at most one item in /” is obvious: if the solution uses one item 
in /, all other items of the solution are in I \ /. 

Consider now UKP. When we talk about solutions, we would formally have to distinguish between 
an item a' e / in the instance and the item copies of a' that a solution V = {xa : a \ a G I, Xa G N} 
uses. In this paper, we however use the expressions “item” and “item copy” interchangeably when 
talking about solutions. As an example, let us consider the sentence “the solution to I uses at most 
one item in /.” It means that the solution contains item copies of items in I, but at most one item 
copy whose corresponding item is in I. To be more precise, the multiset V uses only one item a G I 
with a multiplicity Xa > 0. We have Xa < 1, but Xa" = 0 for all other a” G I, i.e. < 1- 

Similarly, “the solution V uses at most two items in I” means that there are only two item copies 
whose corresponding item(s) are in I: we have < 2. 

The interchangeable use of “item” and “item copy” allows for shorter sentences. Moreover, it is 
based upon 0-1 KP where “item” and “item copy” are in fact identical. 


2.1 A First Approximation 

We present a simple approximation algorithm for OPT(/). Take the most efficient item Omeff := 
argmaxag/ Fill the knapsack with as many copies of UmeS as possible, i.e. take 
l- s(a^ ff) J copies of Omeff- Then the following holds: 

Theorem 5. We have Pq := p(amefr) • [ ^(a^ ff) J — 50 PT(I). The value Pq can be found in time 
O (n) and space O (1). 


Proof. Suppose first that Omefr can greedily fill the knapsack completely. Then p(ameff) • [ ^(a^ ff) J ~ 
OPT(/). Otherwise, one additional item Omeff exceeds the capacity c. Then p{a^es) ■ -|- 

p(amefr) > OPT(/). If p(amefr) < lOPT{I), then pia^es) ■ L s(aleff) ^ - OPT(/) - pia^es) > 
^OPT(/), and the theorem follows. Otherwise p(amefr) • [5 > p{ames) > 50 PT(/), which 

also proves the theorem. 

To determine Pq, we only have to check all items (which can be done in O (n)) and to save the 
most efficient item (which only needs time in O (1)). 

(The proof is taken from p. 232, [^) □ 

Assumption 1. From now on, we assume without loss of generality that e < j and e = for 
K G N. Otherwise, we replace e by the corresponding such that < e < Note that 

log 2 (|) = K holds. 

Similar to Lawler [^, we introduce the threshold T and a constant K: 



1 

2 


1 

2 '^-! 


Po 


( 1 ) 
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and 


1 


1 

2^ 


2 


( 2 ) 


K-.= \ 


41og2(f) + 1 


eT= - 


1 1 


1 


4k + 


T = -- 


1 


8 K + 1 


Pn 


We will see later that these values are indeed the right choice for the algorithm. (A derivation of 
these values is presented in Subsection 5.8.1].) 


3 Reducing the Items 


We hrst partition the items into large(-proht) and small (-profit) items, and only keep the most 
efficient small item: 


lL-.= {aeI\p{a)>T], Is:=I\Il: 


and Oeff := argmax 



p{a) < T 


Theorem 6. The sets Il,Is o,iT'd the item Oeff can be found in time O (n) and space O (n). This is 
also the space needed to save II- 

Proof. Obvious. □ 

Similar to Lawler, we now reduce the item set II. Note that we have OPT(/) < 2Po according 
to Theorem]^ and one item cannot have a profit larger than OPT(I) < 2Po- Hence, the large item 
profits are in the interval [T, 2Po]- We partition this interval into 

:= [2^r,2^+^r) for A: e {0,...,K+1} . (3) 


Note that 







1 

2pu 


Po, 


2^+1 


1 

2 



[Po,2Po) 


For convenience, we directly set := {2Po}. 

We further split the into disjoint sub-intervals, each of length 2^K: 


L('=) •= 

7 

2^T + 7 • 2^K, 

2^T + {-i + l)2^ 


for 7 G 

[0,...,2"+1(k + 1) -l} 

Note that indeed 


holds because 




2^T + {-f + l)2’^K 

7=2'=+1(k+1)-1 = 

2^T 

+ 2'^+^{k 

-h i)2^a: 



§ 

2^T 

+ 2'^+^{k 

+ 1)2^^ ^ 

^ 4k + 1 2«^-i 



= 

2^T 

+ 2^r = 


Similar to above. 

we set : 

= {2Po}. 





(k) 

The idea is to keep only the smallest item a for every profit interval . We will see that these 
items are sufficient to determine an approximate solution. 
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Definition 7. For an item a with p{a) > T, let k{a) G N 6 e the interval such that p{a) G 
and 7 (a) G N 6 e the sub-interval such that p{a) G Let a!^'^ be the smallest item for the profit 

interval L^\ i.e. 

:= argmin |s(a) | a G II and p{a) G for all k and 7 . 

Algorithm shows the algorithm to determine the . They form the reduced set of large items 

^L,red:=UU{«?^} • 

k 7 


As in 21 , we now prove that Iired is sufficient for an approximation. 


Algorithm 1: The algorithm to determine the a. 


(k) 


for A: = 0,..., K do 

for 7 = 0,..., + 1) — 1 do 

L 4^) := 0; 

:= 0 ; 

for a £ II do 

Determine (A:(a), 7 (a)); 

(fc(a))x .X JHa)) 


if s(a;';7)>Ka) = 


= 0 then 


a 7 / := a; 

7(a) ’ 


Output: II,red ■= UfcU7{«7^^} 


Lemma 8. Let 0 < n < c = 1. Then 

OPT ({aeff}, c-v)> OPT {Ls, c-v)-T 


and 


OPT {II, red, V) > 


1-i 


41og2(-) + 1 


OPT {Il,v) 


Proof. For the first inequality, there are two possibilities: either copies of Oefr can be taken such 
that the entire capacity c — u is used. Then obviously OPT ({oeff}, c — u) = 0 PT(/ 5 ,c —u) 
holds. Otherwise, we have similar to the proof of Theorem that OPT({aeff},c — v) + p(aefr) = 

■ P(®efr) + p(aeff) > OPT {Is, c-v). Thus, OPT({oefr}, c-v)> OPT(l 5 , c-v) - p{aes) > 
0PT(/5, c — v) —T. The first inequality follows. 

For the second inequality, take an optimal solution {xa)aei such that OPT {Is, v) = 

ikia)) 

Replace now every item a by its counterpart ofy^a) ™ -^L,red- Obviously, the solution stays feasible. 
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i.e. the volume v will not be exceeded, because an item may only be replaced by a smaller one. This 
solution has the total profit Sos/l Moreover, we have 


> p(a) - ® p(a) 


1 1 1 


4«; + 1 2'"-! 
>2>‘y°-)T /I 1 1 \ / 


p(a)>2'=(“)T 


rj^ 

1 1 1 


4k + 12'"-i 


(k) 

by the definition of the . We get 

OPT(/i,,ed,T^) > E i E 


a&lL 


cl^Il 


1 1 1 
1 - T 


4 K + 1 2«^ 


■p{a)xa 


(1- - OFT {Il,v) 

V 4k + 12 «-V ^ ^ 

0PT(4,n) 


= 1 - 


41og2(-) + 1 


(5) 


(The reasoning is partially taken directly from or close to the one by Lawler in j^.) □ 

Theorem 9. The set lL,ied has 0{h log^ items. Algorithm^^ needs time in 0{n + ^ log^ and 

spaee in 0{j log^ j) for the eonstruetion and for saving lL,red- 

Proof. The number of items a!y \ including the item is bounded by 0 ((k + 1) • (2^+^(k+ 1) — 

1 + 1)) = 0(log ^ log ^)) = 0(^ log^ ^). The space needed is asymptotically bounded by the space 

required to save the a^\ Finally, the running time is obviously bounded by 0(n + ^ log^ ^): the 
values k{a) and 7 (a) can be found in 0 ( 1 ) because we assume that the logarithm can be determined 
in 0 ( 1 ). □ 


Remark 10. If there is one item a with the profit p(a) = 2 Pq, i.e. whose profit attains the upper 
bound, one optimum solution obviously consists of this single item. During the partition of I into 
II and Is, it can easily be checked whether such an item is contained in I. Since the algorithm can 
directly stop if this is the case, we will from now on assume without loss of generality that such an 
item does not exist and that = 0 . 


4 A Simplified Solution Structure 


In this section, we will transform lL,red into a new instance G whose optimum OPT(G, v) is only 
slightly smaller than OPT(/i^i.ed, n) and where the corresponding solution has a special structure. 
This new transformation will allow us later to faster construct the approximate solution. First, we 
define 


:= {a G lL,red I p{a) G = {a G lL,red 


p(a)G 2^T,2 


)fc + l 



Note that the items are already partitioned into the because of the way li^red has been 
constructed. 









Definition 11. Let ai,a 2 be two knapsack items with s{ai) + 5 ( 02 ) < c. The gluing operation © 
combines them into a new item ai © 02 with p{ai © 02 ) = p{ai) +^( 02 ) and s{ai © 02 ) = s(ai) + 5 ( 02 ). 


Thus, the gluing operation is only defined on pairs of items whose combined size does not exceed 


The basic idea for the new instance G is as follows: we first set := Then, we construct 
ai © 02 for all 01,02 G G^^^ (which also includes the case oi = 02 ), which yields the item set 
:= {oi © 02 I oi ,02 G Note that p(oi © 02 ) G [2T,dT) = For every profit 

interval L^\ we keep only the item of smallest size in U which yields the item set G^^\ 
This procedure is iterated for k = 1,..., k — 1: the set G^^^ contains the items with a profit in 
pfc ji, 2^+1 j") — ^(fc) ^ggg ] 'a,)). Gluing like above yields the item set with profits in 

(see Fig. 1 T)). By taki ng ag ain the smallest item in U for 

every the set is derived (see Fig. if^). The item in G^^^ with a profit in is 

(k) —' 

denoted by a\ ' for every k and 7 . 

We finish when G^'^1 has been constructed. We are in the case where lG+^) = 0 ^ = 0) 

and it is explained at the beginning of Section that it is not necessary to construct from 

the items in G^^\ Hence, we also have Og^^^^ = 0. 

Note that we may glue items together that already consist of glued items. For backtracking, we 
save for every which two items in have formed it or whether a^'^ has already been an 

item in Algorithm presents one way to construct the sets G^^\ 


-Ak) 


is in fact the combination of several items in lL,red- The profit and 


Remark 12. One item a 

size of a^'^ is equal to the total profit and size of these items. The a^'^ represent feasible item 
combinations because an arbitrary number of item copies can be taken in UKP. 


The item set 

G := U G(^) 

k=0 

has for e very 0 < u < c a solution near the original optimum OPT {lL,red,v) as shown below in 
Theorem |l4[ It is additionally proved that at most one item of every G^^^ for k G {0,.. ., k — 1} is 
needed. First, we introduce a definition for the proof. 

Definition 13. Let I' be a set of knapsack items with p{a) > T for every a G I'. For a knapsack 
volume V < c and feo G {0,..., k}, a solution is structured for k = ko if it fits into v and uses for 
every fe G {0,..., ko} at most one item copy with a profit in L^^\ We denote by OPT<fcQ (/', v) the 
corresponding optimum profit. 


For instance, the solution for 

OPT<fco (g(°) U ... U U g(^o+^) U U ... U u) 

fits into the volume v, and it uses only one item from every G^^'l for k G {0,..., ko}. It may however 
use an arbitrary number of item copies e.g. in or 
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(a) The items in and The height of every item a corresponds to its size s(a) while its position on the axis 

corresponds to its profit p{a). The axis is partitioned into the profit sub-intervals 2 *T-|-( 7 -|-l) 2 *^Ji'). 



(b) The set together with the newly constructed items in jjik+i) 



(c) The new set after keeping only the smallest item with a profit in For instance, 04 © 04 is kept 

because it is the smallest item in its profit sub-interval 


Figure 1: Principle of deriving from and 
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Algorithm 2: The construction of the item sets 


for A: = 0,..., K do 

for 7 = 0 ,..., + 1) — 1 do 

~(fc) (fc) 

ay := , 

Backtrack(a^*^^) := 

G( 0 ) ;=I( 0 ); 

for A: = 0,..., K — 1 do 

for 7 = 0 ,..., + 1) — 1 do 

for 7' = 7,..., 2 '^~^^{k + 1) — 1 do 


if s{a^'') + s(dY^) < c then 
d := d^y^ © hy^; 

if s(a) < or = 0 then 

;= a; 

Backtrack(al^t^^) := {d!'y\d^^})] 




Q(k+i) 


(fc+i) 


.(fc+i) 


’ "'2'=+1(k+1)- 


.f 


Theorem 14. For v < c and G {0 ,... ,k — 1}, we have 


' fco+l 

u 

k=0 


1 


/cq+I 


OPT<fc J IJ U U jW, J , 7 ) OPT(/z.,red,^^) • 


k=kQ-\-2 


41og2(-) + 1 


Proof. The proof idea is quite simple: we iteratively replace the items in /(*^o+i) i^y their counterpart 
in and also replace every pair of item copies in by the counterpart in ( 7 (fco+i). This 

directly follows the way to construct the item sets G^^'^ presented in Algorithm]^ 

Formally, the statement is proved by induction over kQ. Let A:o = 0. Take an optimum solution 
to G^^'^ U U ... U U U ... U = II,red- For ease of notation, we directly write 

each item as often as it appears in the solution. We have three sub-sequences: 

• Let oi,..., (r/ G N) be the items from G^^^ = in the optimal solution for OPT(/^^i.ed) ^)- 

We assume that rj is odd (the case where rj is even is easier and handled below.) 

• Let a^+i,..., 0 ) 7 +^ G N) be the items from I^^'> in the optimal solution for OPT(l 2 ,_red) v). 

• Let a^,..., d\ (A G N) be the remaining items from U ... U in the optimal solution for 
0 PT(/77 red) This set is denoted by A. As defined above, the total profit of these items is 
written as p(A). 

Figure gl a)] illustrates the packing. (Figure]^ shows the case for general k.) We have 

V v+i 

OPT u) =^p(ai)+ ^ p{dj)+p{A) . (6) 

2=1 j=r}-\-l 
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In the first step, every pair of items d 2 i-i and d 2 i from for i G is replaced 

by a 2 i-i © d 2 i G (see Fig. ^ 'b)). In the second step, every item a 2 i-i © d 2 i is again replaced 
by the corresponding item i©a 2 ') ™ ^ {I)---) LiJi)- hem 

remains unchanged. Moreover, dj from is replaced by the corresponding ■) 
j G {t/ + 1,..., T/ + ^} (see Fig. 1 Note that this new solution is indeed feasible because the 
replacing items are at most as large as the original ones. Moreover, the corresponding items 
and must exist by the construction of Thus, we have a (feasible) solution that 


consists of the item G G^^\ the items and 




p(i) 


m 


G^^\ and the remaining items a'l,... 


in this solution respects the structure of OPT<fcQ(-,r;) for ko = 0. (If rj is even, no 

item in G^^^ is used.) 

Let now d be an item 02*-1 © d 2 i or dj. It can be proved as for Inequality (|^ that 




(7) 


Thus, we have 


LiJ P+« 

OPT<o(g(°)uG«U/(2)u...U/("),^;) >p(a^)+gp(aW ^ p(aWp+p(A) 

j=r)+l 


2 = 1 


HI 


> p{ar,) + 1 - 


+ 1-T 


4iog2(-) + i;^ 


^p(a2i-i © a2i) 


1 


V+i 


> 1 -^: 


41og2(-) + 1 
1 \ 


P(«i)+P(^) 


§ 


1 -i 


41og2(-) + 1^ 
1 


j=rj+l 

V v+i 

^p{di) + p{dj) 

i=p+i 


. 2=1 


41og2(^) + l 


1 - -- 


41og2(^) + l 


+ p(A) 

OPT(/Gred,H • 


OPT 


The statement for /cq = 1,..., k — 1 now follows by induction. The proof is almost identical to the 
case /co = 0 above, the only difference is that there are additionally the items in G^^\ ..., that 

remain unchanged like the items /(*^o+2)^ _ _ _ ^ /H). Only the items in G(^o) and are replaced. □ 


Lemma 15. OPT(G U {oefr}) < OPT(/L,red U Is) < OPT(/l U Is) = OPT(/) < 2Pq holds. 

Proof. G consists of items in lL,red or of items that can be obtained by gluing several items in 
7L,red together. Every combination of items in G can therefore be represented by items in II,red- 
Moreover, we have Oefi G Is. The first inequality follows. Since II,red H II, the second inequality is 
obvious. The last inequality follows from Theorem □ 
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(a) The current solution to 
G™ U • • • U G''“^ U U 

• • • U The structure of 

OPT<fe_i(-,u) is respected, 
i.e. at most one item from ev¬ 
ery G(°’,...,G(''-^^ is used. 


l{k+^) 






34 


ai 


ao 

■© 

33 


34 


34 


34 


(b) The items 
in (j(k) 

are pair¬ 
wise glued to¬ 
gether with the 
possible excep¬ 
tion of one item. 


/(^+2 )./(k)< 






G(o). 


34 

■© 

34 


ai 


ai 


34 


34 


34 


(c) The items in 77''=+^^ U 
7 (fc+i) g^j.g replaced by their 
counterparts in Now, 

at most one item in is 
part of the solution, and the 
structure for OPT<fc(-,u) is 
respected. 


Figure 2: The principle of the proof for Theorem 
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Up to now, we have (only) reduced the original item set / to G U {oefr}. 
Lemma 16. Assume as mentioned in Remark 10 


that = 0. Consider the optimum structured 

solutions to GU {aefr} for ko = k — 1 (see Definition 13). This means that at most one item is used 
from every for /c G {0,..., k — 1}. (The item Oeff has a profit p{aes) < T such that it does not 
have to satisfy any structural conditions.) Then there are two possible cases: 

• One solution uses (at least) two items in G^'^\ This is the case if and only if the optimum for 
Gujaeff} is 2 Pq, and the solution consists of two item copies of the item with p{al)^^) = Pq. 

• Every solution uses at most one item in G^^\ Then, OPT<k-i(G, f') = OPT<k(G, holds 
for all values 0 < fi < c, and there is a value 0 < v < c such that 

OPT<« (G, v) + OPT({aeff} ,c-v) = OPT<,_i (G, v) + OPT ({aeg} ,c-v) 

/ -1 \ k ;+1 

£ 1 


> 1- v; 


OPT(/) - T . 


41og2(^) + l, 

Moreover, OPT<k(G, r») uses at least one item in U U G^'^\ and/or we have 


OPT ({Oeff} ,c-v) > IPq. 


Proof. Note that lL,Ted does not contain any item with the profit 2Pq (see Remark 10). By 
construction, this is still the case for G. Suppose now that one solution to G U {aeg} uses more 
than one item in G^^\ Since items in G^'^^ have profits in [Po)2Po)) only two copies of the item 
can be used, and we have p(aQ^^) = Pq. In fact, 2Po is the maximum possible profit because 
OPT(GU {oeg}) < OPT(/) < 2Po holds as we have seen in Lemma 15 Thus, the “only if” direction 
has been proved. The “if’’-direction is obvious. 

Suppose now that every structured solution to G U {oeg} for kg = k — 1 uses at most one item 
in G^'^f Thus, OPT<^_i(G,P) = OPT<«(G,U) holds for all 0 < P < c. 

Let V < c now be the volume the large items Iz occupy in an optimum solution to I. Then 
obviously OPT(/) = OPT (l£, v) -f OPT (I5, c — v) holds. We have the following inequality: 

OPT<, (G, v) + OPT({aeg} ,c-v) = OPT<,_i (G, v) + OPT({ueg} ,c-v) 
e 1 


Thm. 1141 
> 


Lem. | 8 ] 
> 


> 


1 - 


1 - 


41og2(f) 

+ 1 

e 1 


41og2(f) 

+ 1 

e 1 


41og2(f) 

+ 1 

e 1 


41og2(f) 

+ 1 


I OPT(/i,,ed,^^) + OPT({aeg},c-i;) 

K+1 

I OPT (/i,u) +OPT (Is, c-u)-r 




(OPT {II, v) + OPT {Is, c-v))-T 


K+1 


OPT(/) - T . 


( 8 ) 


For the final property, suppose that no item in G^'' U G^'” U G^'^) is used in a solution for 
OPT<k(G, u). Then we have 

m 1 


k—3 

OPT<k (G, v) < ^ max 
fc=o 


[p{a) I a G G(^)} < ^ 2 • 2^r < 2'^-^r “ -Pg 
k=0 ^ 
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On the other hand, Inequality Q together with (1 - 
OPT<, (G, v) + OPT ({aeff} ,c-v)> 

i (^1-0 OPT(I) - ^ePo > (l 

£<1/4 Q Q 

> -OPT(/) > -Po . 


5 )^ > (1 — A; • d) for 5 < 1 yields 


£ K + 1 \ 

"41og2(f) + lyl 


OPT(/) - T 


^ OPT(/) - ^eOPT(I) 


Hence, OPT({aeff} ,c — v) > |Po holds. The final property of the second case follows. □ 

Definition 17. Take r p(a^ff) ~l ®efr- If their total size is at most c, they are glued together to 

^eff—c* 

Obviously, Oeff-c consists of the smallest number of items Oeff whose total profit is at least 
Moreover, Oeff-c is a large item. 


Definition 18. Take a knapsack volume v < c. Consider the following solutions to GU {aeg-c} of 
size at most v: 


• They are structured for k = k, i.e. they use for every k ^ {0,... ,k} at most one item in 

• They additionally use the item Oeg-c at most once and at least one item d G U 

GWu{aeff-c}. 

Hence, these solutions have a profit of at least p{d) > 2'^~^T = ^Pq. These special solutions are 
called structured solutions with a lower bound (on the profit). 

The optimal profit for such solutions of total size at most v is denoted by OPTst(GU {oeg-c}, v)- 
If V is too small such that such a solution does not exist, we set OPTst(GU {oeg-c},!’) = 0. 


Theorem 19. In the second case of Lemma 16. there is a value 0 <v <c such that 

K +1 

I OPT(/) - T . 


OPTst {G U {aeff-c} , v) + OPT ({aeg} , c - u) > ( 1 - ^ ^ 


41og2(4) + l 


Proof. Like in the proof of Lemma [I^ let v' be the volume the large items Il occupy in an optimum 
solution to I so that OPT(/i,P) + 0PT(/5,c — P) = OPT(/). Consider an optimum solution 
for OPT<k(G, and suppose that it does not use any item in U U G^'^f Lemma 


of Oeg can be replaced by Oeg-c, and c — v > s(aeg_c)- We set v := v + s(aeg_c)- Note that 
OPTst(GU{aeg-c},?;) > OPT<,(G,P)+p(aeg-c). Moreover, OPT<,(G,P) = OPT<,_i(G,P) 


16 


states that OPT({aeg}, c — v') has a profit of at least jPq- Thus, a subset of the item copies 
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holds because we are in the second case of Lemma We get the following inequalities: 


OPTst {G U {Oeff-c} , v) + OPT ({Ueff} , C - u) 

> OPT<k (G, v) + p(aefr-c) + OPT ({oefr} ,c-v' - s{aes-c)) 

= OPT<,_i(G,u') + OPT({aefr},c-u') 

Thm.nH / e 1 

- \^~ a ] -72TTT OPT(lL,red,^^) +OPT({aefr},c-u) 

V 4iog2(-) + iy 

Lem. m / e 1 \ 

S OPT (/.,.')+OPT (/s,.-«')-!■ 

^ (' - ibSifirr)''O'-')+OPT-T 

Note that OPT({aeff}, c — v) is well-defined—and therefore the entire chain of inequalities feasible— 
because c — v = c — v' — s(aeff-c) > 0. 

Suppose now that the optimal solution uses at least one item in u u GG) . We can 

then directly set v := v', and the proof is similar to the first case above. 

Roughly speaking, a solution in the first case of this proof satisfies the lower bound of the 
theorem and uses at most one item in every G^^\ but no item in GG~‘^\gG~^^ or GG). This 
implies that enough items Oefi are part of the solution such that a subset of them can be replaced 
by Oeff-c- □ 


So far, we have not constructed an actual solution. We have only shown in Theorem 19 that 
there is a solution to G U {oeff-c} U {oeff} that is close to OPT(/) and that is a structured solution 
with a lower bound. 


Theorem 20. The cardinality of G^^'l is in O(^log^), i.e. G has 0(Mog^ items. Algorithm^^ 
constructs G in time 0(^log^(^)) and space 0(jlog^ ^), which also includes the space to store G 
and the backtracking information. The item Oeff-c can he constructed in time 0(1). 


Proof. The statement for Oeff-c is trivial: the number of items Oeff-c to glue together can be 
determined by division. 

The number of items in G^^^ and G can be derived like the number of items in lL,ied in Theorem 
The running time of Algorithm is obviously dominated by the second for-loop. It is in 



(k) 

The space complexity is dominated by the space to save the d\ ^ and the backtracking information, 
which is again asymptotically equal to the number of items in G. □ 
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5 Finding an Approximate Structured Solution by Dynamic Pro¬ 
gramming 

The previous section has presented three cases: 

1. The instance I has one item of profit 2Pq: return this item for an optimum solution, and 


OPT(/) = 2Pq (see Remark 10). 


2. If this is not the case, and G has one item of profit Pq and size at most §, two copies of this 


item are an optimum solution to G U {oefr} (see Lemma 16). Undoing the gluing returns an 
optimum solution with OPT(/) = 2i-o- 

3. Otherwise, there is an approximate structured solution to G U {aeff-c} U {oeff} with a lower 


bound (see Theorem 19). 


The first two cases can be easily checked, which is the reason why it has not been necessary to 
construct the set We will from now on assume that we are in the third case: a solution uses 

at most one item from every G^^^ for k G {0,as well as Ueff-c at most once. At the same 
time, at least one item a G U U G^'^^ U {oefr-c} is chosen. (See Definition 
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We use dynamic programming to find for all 0 < u < c the corresponding set of large items 
U C GU {aeff_c} with s(U) < v. For convenience, let := {aefr_c}- We introduce tuples 

(p, s, k) similar to Lawler |^. For profit p with 0 < p < 2Po and size 0 < s < c, the tuple (p, s, k) 
states that there is an item set of size s whose total profit is p. Moreover, the set has only items in 
G(*^) u • • • U and respects the structure above. 

The dynamic program is quite simple: start with the dummy tuple set fG+‘^) ■_= |(o, 0, k + 2)}. 
For fc = K + l,...,K — 2, the tuples in are recursively constructed by 

■.= [{p,s,k) I (p,s,A: + l) gF(^+^)} 

U |(p + p{a),s + s(a), k) \ (p, s, /c + 1) G G G^^\s + s(a) < c| . 

Note that (0,0, A: + 1) G which guarantees that also contains the tuples (p(a), s(d), k) 

for a G G^^) if A: G {k + 1,..., K — 2}. For A; G {k — 3,..., 0}, this tuple (0, 0, A: + 1) is no longer 
considered to form the new tuples, which guarantees that tuples of the form (p + p(a), s + s(a), k) 
for a G G^^^ have p, s 7 ^ 0. The recursion becomes 

F(^) := {(p,s,A:) | (p,s,A: + l) G 

U |(p + p(o), s + s(a),k) I (p, s. A; + 1) G \ {(0,0, A; + 1)} , a G G^^\s + s(a) < c| . 

The actual item set corresponding to (p, s, k) can be reconstructed by saving backtracking information. 


Definition 21. A tuple {p 2 ,S 2 ,k) is dominated by {pi,si,k) if P 2 < Pi and S 2 > si. 

dominated tuples (p, s, A: + 1) are now removed from before F^^^ is constructed. 


As in 


21 


This does not affect the outcome: dominated tuples only stand for sets of items with a profit not 
larger and a size not smaller than non-dominated tuples. A non-dominated tuple (p, s, k) is therefore 
optimal, i.e. the profit p can only be obtained with items of size at least s if items in G^^\ ..., 
are considered. 
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Lemma 22. A tuple {p,s,k) G stands for a structured solution with a lower bound (see 
Definition 18). Therefore, we have p > 2'^“^T if p > 0. For every v < c, there is a tuple 
{p, s, 0) G F^ with p = OPTst(G' U {aeff-c}, and s < v. 


Proof. This lemma directly follows from the dynamic program: tuples use at most one item from 
every GW. For k G {k — 2 ,...,K + l},a tuple with p > 0 represents an item set that uses at least one 
item in ..., and such an item has a profit of at least 2^“^T. Tuples for k < n — 3 with 

p > 0 are only derived from tuples that use at least one item in G^'^~‘^\ ..., If dominated 

tuples are not removed, the dynamic program obviously constructs tuples for all possible structured 
solutions with a lower bound, especially the optimum combinations for every 0 < u < c. Removing 
dominated tuples does not affect the tuples that stand for the optimum item combinations so that 
the second property still holds. □ 


While the dynamic program above constructs the desired tuples, their number may increase 
dramatically until F)^'> is obtained. We therefore use approximate dynamic programming for the 
tuples with profits in [|Tb) 2Po]- This method is inspired by the dynamic programming used in 
(see also pp. 97-112]). 

Dehnition 18 and Lemma 22 state that a tuple (p, s, k) with p > 0 satishes p > 


15 


Apart from (0,0, A:), all tuples have therefore prohts in the interval [ 2 '^ ^T,2Po] — [\Po-,‘^Pq\ = 
[2^“^T,..., 2'''''^r]. We partition this interval into sub-intervals of length We get 


r N 

= y ^ . 2'^-2iF, 2'^-2r +{(, + 1)2^-'^K] U {2Po} 

5=0 

=: U u 

5=0 

for (^0 := 7(k -|- 1 ) 2 ^+^ — 1 . (A short calculation shows that 2 ^“^T -|- (^o + l) 2 '^“^Ar = 2Po-) The 
approximate dynamic program keeps for every ^ G {0,... ,^o + 1} only the tuple (p, s,fc) with 
p € ' that has the smallest size s. The dominated tuples are removed when all tuples for k have 

been constructed. The modified dynamic program is presented in Algorithm]^ and shown in Figure]^ 
The sets of these non-dominated tuples are denoted by For convenience, {p{f), k) G 

denotes the smallest tuple with a proht in . We again save the backtracking information 

during the execution of the algorithm. 

Lemma 23. Let F))^) be the set fro m Algorithm^before the dominated tuples are removed. 

A tuple (p, s, k) G for k = K + 1 ,... ,0 stands for a structured solution with a lower bound. 
Therefore, we have p > 2^~‘^T if p > 0. This is also true for (p, s, k) G . 

Proof. The proof is almost identical to the one of Lemma In fact, the proof is not influenced by 
keeping only the tuple of smallest size in every profit interval □ 


Theorem 24. Let k G {0,..., k -|- 1}. For every (non-dominated) tuple (p, s, k) G F^P, there is a 
tuple (p, s, k) G such that 


P > 


e 1 

41og2(f) + 1 


K—fc +1 

p 


and s < s . 
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Algorithm 3: The approximate dynamic programming 

jjiK+2) {( 0 , 0 ,k + 2 )}; 

Backtrack(0,0, k + 2) := 0; 

for k = K + 1,... ,0 do 

:= 0 ; 

for {p{^), s{^),k + 1 ) G do 

:=Z)Wu{(p(0,s(0,^)}; 

Backtrack(p(^), s(^), k) := Backtrack(p(^), s(.^), A: + 1); 

for a G do 

for (p, s, A: + 1) G \ {(0, 0,/c + 1)} do // Construction of new tuples 

{p',s',k) := {p + p{a),s + s{a),k); 

Determine for (p', s', k) such that p' G 
if s' < s(.^') or (p(C0)= 0 then 

// Only new tuples of smaller size are kept 
jjik) :=DW\{(p(0,s(0,fc)}; 

{p{C'),s{^'),k) := {p\s',k)] 

Backtrack(p(^'), s(^'), k) := ((p, s, A; + 1), a); 

1^ [zi(D:=ziWu{(p(0,KO,fc)}; 

if k > K — 2 then 

// Construction of (possible) tuples (p(a), s(a), A:) for k > n — 2 
Determine for p(a) such that p(a) G 
if s(a) < s(^') or {p{i'),s{^'),k) = 0 then 

:=DW\{(p(0,s(0,fc)}; 

{p{0,s{i'),k) := {p{a),s{a),k)] 

Backtrack(p(^'), s(^'), A:) := (a); 

[ [ Zl(D:=z)(Du{(p(0,KO,fc)}; 

Remove dominated tuples from 
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<>^z 



s 


(a) The tuples in 



S2 < Si S 


(c) Only the tuple of smallest size is kept for every 
which yields . . 



s 


(b) The new tuples are constructed with the items 
in 



S <S2 S 


(d) ... and removing the dominated tuples yields 


Figure 3: The principle of the approximate dynamic programming 











Proof. This statement is triviai for {p, s, k) = (0, 0, k) because (0, 0, k) G (this tupie is never 
removed in the construction of and 

Suppose now that (p, s, k) / (0, 0, k). The theorem is proved by induction for /c = k + 1,..., 0. 
The statement is evident for k = k + 1. If a^s-c exists (i.e. enough copies of a^s can be giued 
together without exceeding the capacity c), then we have 

^(k+1) ^ ^(k+ 1) ^ 0, K + 1), (p(aefr-c), s(aeff-c), « + 1)} • 


If Oeff-c does not exist, then we have = {(0, 0, k + 1)}. 

Suppose that the statement is true for A; + 1,..., k + 1. As defined in Lemma 
before the dominated tuples are removed. Let (p, s, k) G F^^'> 
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ij{k) 

is the set 


There are two cases. In the first case, we have (p, s, A; +1) G . By the induction hypothesis. 



profit, we have 


p 


> P2 > Pi - 2^ Pi 


■)AC —2 


1 - 


K 


Pi 


’-y-' . 


1 - 


4Iog2(f) + 1 


>p- 1 - 


Lem. 1231 
> Pi • 


■)K —2 


1 - 


K 


2 '"- 2 T 


41og2(f) + 1 


K — k + 1 


The lower bound on the profit is therefore true for (p, s, k). We have s < S 2 < s\ <s for the bound 
on the size (see also Fig. ^'c) |. 

Consider now the second case where {p,s,k) G F^^\ but (p, s. A: + 1) ^ Therefore, 

(p, s, k) is a new (non-dominated) tuple with (p, s. A;) = (p + p(a), s + s(a), k) for the right item 


a G and tuple {p,s,k + 1) G By the induction hypothesis, there must 

(pi, si. A: + 1) G such that pi > p(l — j -—and si < 5 (see Fig. 

* l°g 2 (,E J + l 

the following inequality holds: 


3 e a tuple 
a)). Thus, 


K—(A:+l)+l 


Pi + p(a) > p(a) + p • 1 - 


41og2(f) + 1 


> (p(a) + p) 


1 - 


41og2(f) + 1 


K;-(fc+l)+l 


= P' 


1 -L 


1 


k-(A:+1)+1 


41og2(^) + l 


There are two possibilities: either k > n —2, i.e. p(a) > 2^“^r holds, and pi +p(a) > 2^“^T directly 
follows. Otherwise, we have k < k —3. Then, the identity (p, s, k) = (p + p(a), s + s(a), k) / (0, 0, k) 
implies that (p, S, A; + 1) ^ (0, 0, A; + 1) holds because the tuple (0, 0, A: + 1) is not used to form 
any new tuple in and therefore in This again implies that pi 7 ^ 0 and therefore 

Pi + p(a) > Pi > 2'^“^T as seen in Lemma 23 
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IV 


o 




s 

IV 


■o 


+ 

*07 


(p.s, k) 






(Pl.- 

(P2,S2,k) 1 

--• 

h, k) 

1 







S2 < Si < S S 


(a) Since (p, s, fc+ 1) £ ^’('=+1)^ there must be a corre- (b) By construction, there must be a tuple (p2, S2, fe) € 
sponding tuple (pi, Si, fe + 1) £ by the indue- with a profit in the same interval as (pi, si, k). This 

tion hypothesis whose profit can be bounded from makes it possible to bound p 2 from below, 
below. 


■o 

IV 

S 

IV 


o 

s 

I 

X- 




[P.s.k) 




(P.s.^) 



(Pi.- 

(P2,S2,/t) 1 

a, k) 

1 








< S2 


(c) There may be a tuple (p, s, k) £ that domi¬ 
nates (p 2 , S2,k). Since p > P 2 , the bound still holds. 

Figure 4: The first case of the proof for Theorem 24 we have {p, s, k) G and also (p, s, /c -|- 1) G 
i?(fc+i). We set g := (1 - I 


4log2(f)-fl^ 


22 












































Thus, there is an index such that pi+p{a) e Similar to above, the tuple (pi+p(a), si + 

s(a), k) is formed during the construction of (see Fig. I T)). It may only be replaced by a 
tuple of smaller size. Hence, there must be (p2, S 2 ,k) G with p 2 G Let (p, s, k) G 


be the tuple that dominates {p 2 , S 2 , k) (see Fig. f|[c) ). We get 

pi+p(a)^0 


2 


p 


> 


P2 > Pi +p{a) -2'^ 


= {pi + p{a)) ■ { 1- 


K 


Pi +P{a) 


pi+p(a)>2''-2r 

> {pi+p{a)) 


■)/€—2 


K 


> 


P- 


1 -^- 


2'"-2T / 

\ K—k+1 


{pi +p{a)) 


1-i 


1 


41og2(^) + 1 


41og2(7) + 1 


We have similar to above s < S2 < si + s(a) < S + s(a) = s for the bound on the size (see also Fig. 


c)|. 


□ 


Remark 25. As can be seen, the proof of Theorem 24 is only possible because it is guaranteed 
that Pi or Pi + p{a) is at least 2'^“^T. In fact, this is achieved by the construction of the glued 


item set G with its structured solution (Theorem 14). Hence, we can prove Lemma 16 and with 
the introduction of Oeff-c, we have the structure property of Definition |18| with a corresponding 
solution (Theorem 19). This shows that pi > or pi + p{d) > 2'^~‘^T (see also Lemma 


22 


and 


. Without the structure, a dynamic program like Algorithm would also have to generate tuples 
{p, s, k) with p < 2'^“^r for k < k — 2>. Hence, we would need for the same approximation ratio 
proht sub-intervals like with a smaller length than 2'^~^K, and we would have to save more 

tuples. Both would increase the asymptotic running time and space complexity as can be seen in 


the proof of Theorem 27 


Corollary 26. For every v < c, there is a tuple (p, s, 0) G such that s < v and 


P > 


1-i- 


1 


K+1 


41 og 2 (-) + 1 


OPTst (Gu{aeff-c},^^) 


Proof. Lemma 22 states that there is a tuple (p, s, 0) G with p = OPTgt(G U {oeff-c} , v) and 
s < V. Theorem |24| implies that there is a tuple (p, s, 0) G with the desired property. □ 


Theorem 27. Algorithm^ constructs all tuple sets for /c = k -|- 1,..., 0 in time 0{^ log^ ^). 
The space needed for the algorithm and to save the as well as the backtracking information is 
in 0{jlog^ ^). 

Proof. Let us first bound the space complexity. The profit interval [^Po,2Pq] is partitioned into 
O(^o) intervals The set saves at most one tuple with the corresponding backtracking 

information for every or the information that a tuple does not exist. Thus, the space 

needed for all and the corresponding backtracking data is in 0(k ■ ^o) = 0{k ■ (/t2'')) = 

0(log(^) • (log(^)^)) = 0(^ log^ ^). All other information of the algorithm is only temporarily saved 
and needs 0(1). 
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(a) Since {p, s, fc +1) ^ there must be an item (b) The tuple (pi + p(a), si + s(a), k) is constructed 

d such that (p, s, k) = (p+p(a), s + s(a), k) for a tuple during the execution of the dynamic program. 

(p, s, fc + 1) £ By the induction hypothesis, 

there must be a tuple (pi, Si, fc + 1) G whose 

profit can be bounded from below. 



(c) As in the first case, there must be a tuple (p, s, k) G 
whose profit can be bounded as desired. Here, (p 2 ,S 2 ,fc) 
is not dominated, i.e. (p, s, k) = (p2, S2,k). 


Figure 5: The second case of the proof for Theorem 24 we have {p, s, k) G but (p, s, A; + 1) ^ 
i?(fc+i). We set g := (1 - 


4log2(f)+H 
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The loops dominate the running time. Apart from removing the dominated tuples, they need in 
total 

O {k • (^0 + 6 • . 1 ^^g 



As stated in and [IT| Lemma 5], non-dominated tuples {p, s, k) can be removed in linear time in 
the number of tuples if the tuples are different and sorted by profit. This is the case because every 
tuple in is stored in an array sorted according to the corresponding The total time to remove 
the dominated tuples from all is therefore in 0(/t • ^o) = log^ i), which is dominated by 
the overall running time. □ 


6 The Algorithm 

We can now put together the entire approximation algorithm. 


Algorithm 4: The complete algorithm 
Input: Item set I 
Output: Profit P, solution set J 
Determine Pq and define T, K ; 

Partition the items into II and 1$ and find Oefr; 
if Item a with p{a) = 2Pq found during the partitioning then 
1^ return 2Pq, {a}; 

Reduce II to lL,red with Algorithm 1 
Construct G with Algorithm and me item Oeff-c; 

if p{a\f^) = Pq and s(aQ^^) < ^ then 

Recursively undo the gluing of ajj ^ to get the item set J . Let J be the set consisting of 
two copies of every item in J'] 
return 2Pq, J ; 

Construct with Algorithm I the tuple sets ..., 

Find (p, s,0) G such that 

P := p + OPT ({oeff}, c - s) = max(p,^^, Q)g^(o) p' + OPT ({aefr}, c - s'); 

Backtrack the tuple (p, s, 0) to find the corresponding structured solution with a lower bound 
J' C G U {Oeff-c}; 

Recursively undo the gluing of all a G J' and add these items to the solution set J; 

Add the items of OPT({aefr}, c — s) to J; 

return P, J ; 


Theorem 28. Algorithm^ finds a solution of value at least (1 — e)OPT(/). 

Proof. The algorithm returns a feasible solution: (p, s,0) represents an item set of size s. If items 
d € G derived from gluing are part of the solution, their ungluing does not change the total size nor 
the total profit (see Remark 12). 
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We prove the solution quality. First, the algorithm considers the two special cases listed at the 
beginning of Section]^ Each of them returns a solution of profit 2Pq so that OPT(/) = 2Po (see 
Theorem]^ and Lemma [T^. If the special cases do not yield a solution, we are in the third case. 


Let V be the volume from Theorem 
with s < V such that 
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Corollary 26 guarantees the existence of one (p, s, 0) G 


P > 


1 - -- 


K+1 


41og2(|) + l 


OPTst (G'U{oefr-c},^;) 


Moreover, we have OPT({aefr}, c — s) > OPT({oeff}, c — v) because c — s > c — v. Thus, the following 
inequality holds for this (p, s, 0): 


p + OPT ({aefr},c - s) 

^ e 1 


re+l 


> 


> 


1 - V 


41og2(f) + 1 


OPTst (G U {Ueff-c} , v) + OPT ({Ueff}, c - v) 


1 - -- 


1 


K-\-l 


Thm.IlQl 

> 


> 


1 - V- 


41 og 2 ( 4 ) + l, 
e 1 


41og2(4) + l 


I (OPTst (G U {aeff-c} , v) + OPT ({aes}, c - u)) 

2 kH -2 / 1 \ 


1 - 


\ 2k+2 


1 - 


41og2(-) + 1/ 
e 2 k + 2 \ 

41og2(f) + 1 i 


OPT(/) - T 


OPT(I) - ^ePo 


> 


1 - T 


(log2(f ) + l) 


log2(4) + 1 
(l-e)OPT(/) . 


OPT(/) - -eOPT(/) 


Taking the maximum over all (p, s, 0) G therefore yields the desired solution. Note that we 
have used (1 — 6)^ > (1 — fe • (5) for 5 < 1. □ 

Remark 29. The total bound on the approximation ratio is mainly due to the exponent 2k + 2, i.e. 


that we make the multiplicative error of (1 — 


4 log2(f)+l 


) only 2k + 2 times. Such an error occurs 


when I is replaced by I L,red at the beginning (Lemma , in each of the k iterations in which G is 
constructed (Theorem |14[), and in k + 1 of the k + 2 iterations of the dynamic program (Theorem 
24 and Corollary [26| ) . The error of the dynamic program can be bounded because the structured 
solution with a lower bound has at least one item of profit at least 2'^~‘^T (see the second property 
of Definition and Remark 25). 

Theorem 30. The algorithm has a running time in 0{n + ^ log^ 4^ needs space in 0{n + 

Proof. Determining Pq, constructing II and Is as well as finding a^s can all be done in time and 
space O (n) as stated in Theorems and The definition of T and K in time and space O (1) is 
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obvious. It is also clear that an item p{a) = 2Pq can directly be found during the construction of Ii, 
such that the first if-condition does not influence the asymptotic running time. 

Algorithm returns the set II,red in time 0{n + ^ log^ and space 0{^ log^ (see Theorem 

i- 

Algorithm |2| constructs the and G in time 0{-^ log^ and space 0(^ log^ as explained 
in Theorem |20[ which clearly dominates the construction of fleff-c in O (1). 

The second if-condition can be checked in 0(1). The running time for undoing the gluing will 
be determined at the end of the proof. 


Algorithm 


3 constructs the sets D^^'> in time 0{X log^ and space 0(^ log^ (see Theorem 27). 


For one tuple Xp' ■, 0), the corresponding OPT ({oeff}, c — s') can be found in O (1) by computing 

•p(oefr). Thus, finding the best tuple (p, s, 0) can be done in 0(|Oi°i|) = O(^o) = 0{^ log ^). 
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Since only the currently best tuple (p, s, 0) has to be saved, the space needed is in O (1). 

The backtracking for the tuple (p, s,0) needs time in 0 (k) = O(log^): the backtracking 
information Backtrack(p', s', k) for A; = 0,..., k -|- 1 states whether the tuple was formed by adding 
an item a G and with which tuple (p", s" ,k + l) to continue. Hence, the item set J' also has at 
most 0(log ^) items in GU {aeff-c}, which bounds the storage space needed. 

To conclude, the time and space for the ungluing still have to be bounded. Consider one item a G 
G. The backtracking information Backtrack(a) returns two items (01,02) (with 01,02 G lL,red U G) 
on which the backtracking can be recursively applied. The recursive ungluing of the items can be 
represented as a binary tree where the root is the original item a and the (two) children of each 
node are the items (o', o") returned by the backtracking information. The leaves of the tree are the 
original items in II, red- This binary tree obviously has a height in 0{k) because the children {a', a”) 
for one a G are in U lived- A binary tree of height 0 (k) = 0(log M has at most 0(M 

nodes. The backtracking or ungluing of a can therefore be done in time and space 0{-^), which 
also includes saving the items a G Ii^red of which a is composed. Since J' has 0(log ^) items, the 
original items Ii^ied of the approximate solution can be found in time and space 0(^ log ^). This 

(k) 

also dominates the time to undo the gluing of Oq ' should the body of the second if-condition be 
executed. 

Similar to above, the number of items Oefi for OPT ({oefi}, c — s) can be found in O (1). To sum 
up. Algorithm has the stated running time and space complexity. □ 


7 Concluding Remarks 


The most important steps in this algorithm are the creation of the item set G by gluing and the 
introduction of aefi-c- This guarantees the existence of an approximate structured solution with a 


lower bound (see Definition 18). Therefore, the approximate dynamic program has to store less 
tuples (p, s, k) than in the case without the structure. 

We have extended our algorithm to the Unbounded Knapsack Profit with Inversely Propor¬ 
tional Profits (UKPIP) introduced in |11| . Here, several knapsack sizes 0 < ci < ... < cm = 1 are 
given, and the profit of an item counts as Pj/ci if packed in ci. The goal is to find the best knapsack 
size and corresponding solution of maximum profit. UKPIP is used for column generation in our 
AFPTAS for Variable-Sized Bin Packing |10| where several bin sizes are given and the goal is to 
minimize the total volume of the bins used. The faster FPTAS for UKPIP yields a faster AFPTAS 
for Variable-Sized Bin Packing |20|. 
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There are interesting open questions. As stated in Subsection the space complexity is a 
more serious bottleneck than the running time. Recently, Lokshtanov and Nederlof |22| showed 


that the 0-1 Knapsack Problem and the Subset Sum Problem have a pseudo-polynomial time 
and only polynomial space algorithm. Subset Sum is a special case of the Knapsack Problem 
where the proht of an item is equal to its size, i.e. pj = Sj. Moreover, it was shown that Unary 
Subset Sum is in Logspace [3 13 . Gal et al. described an FPTAS for Subset Sum whose space 
complexity is in O(^), i.e. which does not depend on the actual input size, and whose running time 
is in 0(^n(n + logn -f log ^)). Can any of these results be further extended to improve the space 
complexity of an UKP FPTAS? 

Finally, it is open whether the ideas presented in this paper can be extended to the normal 0-1 
KP or other KP variants as well as used for column generation of other optimization problems. The 
currently fastest known algorithm for 0-1 KP is due to Kellerer and Pferschy [16| - [l^ . We mention 
in closing that by using the same approach similar improved approximation algorithms can be 
expected for various Packing and Scheduling Problems, e.g. for Bin Covering, Bin Packing with 
Cardinality Constraints, Scheduling Multiprocessor Tasks and Resource-constrained Scheduling. 
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